Panoramic Audio and Video: towards an Immersive Learning experience

Historically, learning has been seen as part of the human process to achieve a status, income and a better future. Traditionally students travel to attend lectures or participate in brainstorming and training sessions benefiting from the interaction afforded by the physical learning environment, the classroom, and the wider University facilities. However, financial pressures arising from national and global competition are putting pressure on Universities to find more cost-effective ways of delivering education. While distant learning has been always there, it is clear that this approach needs substantial improvements in order to deliver a similar learning experience to physically attending a university. To these end, this paper presents a novel immersive telepresence system that allows remote or distant students to customize their virtual presence at seminars and lectures based on a 360° panoramic video projected onto a 180° curved projected screen (an immersive shell). Audio is also collected with 3D information in order to be reproduced more naturally at the remote location. The arrangement is intended to provide online learners with a more faithful replication of human perception, akin to what they would experience in a real learning environment. Technically, to do this we use a 360° mirror to capture the lecture room scene and transmit it to remote locations where it is reconstructed the original image from spherical to Cartesian coordinates providing a novel but natural immersive experience. This paper describes the motivation, model, computational architecture and our main findings.